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Executive Summary 


The FCC and the North American Numbering Council (NANC) charged the 
Numbering Oversight Working Group (NOWG) with compiling and delivering an 
annual performance report of the Pooling Administrator (PA). The PA’s annual 
performance assessment is based upon: 


e 2014 Performance Feedback Survey for PA and RNA (Routing Number 
Administrator) 

e Monthly reports 

e Annual Operational Review 

e NOWG observations and monthly interactions with the PA 


The PA’s rating for the 2014 performance year was determined by the NOWG to 
be Met. This rating is defined below: 


Performance was competent and reliable 


Decisions and recommendations were within requirements 


The Met rating was given to the PA for their consistency in meeting and often 
exceeding all requirements related to Pooling Administration and p-ANI 
Administration. 


The 2014 survey results revealed a consistent level of satisfaction that 
respondents attributed to the responsiveness and expertise exhibited by the PA 
and RNA personnel throughout 2014. 


In 2014, the PA continued to consistently perform its required responsibilities. 
Highlights included: 


e The PA processed 139,181 Part 3s, which was the highest quantity of Part 
3s processed since the start of Pooling. 


e System Development for the new PAS scheduled for deployment in 
January 2015. This included testing and developing documentation by PA 
personnel while continuing to perform their daily work activities. 


e Maintained PAS and RNAS availability at 99.98%. 
e Secured a blanket safety valve waiver in California to enable the PA to 


replenish the pool with resources in the 415 NPA until the effective date of 
the new 628 NPA overlay. 
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Section 1.0 Performance Review Methodology 


The annual PA Performance Evaluations are a summary of significant events 
that were accomplished during the 2014 performance year. In addition to the 
annual performance review survey process, the NOWG’s interactions with the PA 
included the following: 


Monthly NOWG/PA status meetings 
Annual operational review 

PA NANC reports 

Interaction with the industry 


The NOWG introduced a new streamlined survey process for 2014 to stimulate a 
higher survey response rate from entities. The questions were revised anda 
single ‘Comment’ section was created at the end of the survey to consolidate the 
participants’ comments. The NOWG received NANC concurrence to modify the 
rating categories from the five categories previously used: Exceeded, More Than 
Met, Met, Sometimes Met, Not Met, to two categories: Met or Not Met. 


The NOWG also used this same “Met or Not Met” methodology for the overall 
performance evaluation rating. 


The following chart provides the definition of the two rating categories: 


Satisfaction Rating Used when the PA... 


Performance was competent and reliable 
Decisions and recommendations were within 
requirements 


NOT MET e Performance was unreliable and commitments 
were not met 
e Decisions and recommendations were 
inconsistent with requirements 


The NOWG will present the draft report to the FCC and the PA. The final report 
will be presented to the NANC for endorsement and then forwarded to the FCC. 
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Section 2.0 PA Reports 


2.1 PA Annual Report 


The annual report prepared by the PA is a requirement in the Pooling 
Administrator Technical Requirements document. The status of pooling and 
Pooling Administration should be reported in the annual report. Part of the 
NOWG's annual performance review process is to review the annual report. Ata 
minimum, the annual report is required to contain the following information: 


e Brief description of the PA 

e Highlights/significant milestones reached during the previous year 

e Identification of existing and potential pooling areas 

e Aggregated total, by pool, of the service providers participating in the 
pooled area 

e Forecast results, as well as a review of forecasts vs. actual past block 
activations 

e System and performance metrics 

e Status of required transferable property 

e Industry issue identification/feedback 

e Volume of reports produced aggregated by regulatory agency, NANC, 


NANPA, and service providers 
e Additional informational offerings 


The PA provided the NOWG with an opportunity to review the draft copy of the 
2014 Annual Report. During the on-site operational review in Concord, California 
on March 31, 2015 - April 1, 2015, the PA staff reviewed the 2014 highlights 
which were also included in the annual report. 


Overall, the annual report provides a comprehensive snapshot of pooling and the 
PA for 2014. The PA 2014 Annual Report was filed with the FCC and is posted 


for general availability on the PA’s website at www.nationalpooling.com. 
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2.2 PA NANC Report 


The PA reported its monthly numbering activities to the NANC and the NOWG. 
Additionally, the PA made presentations at four NANC meetings in 2014 (March, 
June, September, and December). The PA reported the status of thousands- 
block pooling administration, p-ANI activities, and events affecting the 
performance of the PA, which included the following: 


e Volume of pooling assignments, donations, and applications processed 
e Codes opened to replenish pools and establish Location Routing Numbers 
(LRNs) 

Pools with less than six months inventory vs. forecasts 

Summaries of monthly reports to the FCC 

Number of blocks reclaimed 

Percent availability of PAS and RNAS 

Status and implementation of change orders 

Updates to PAS 

Updates to the PA website 

p-ANI summary 


2.3 NOWG Monthly Reports 


Throughout 2014, the NOWG and PA followed a standing agenda during the 
scheduled monthly calls. The PA provided monthly performance reports that 
were reviewed during the monthly calls with the NOWG. The quality and content 
of these reports provided the NOWG with valuable insight into the operations of 
the PA. Some of the standing agenda topics include: 


Application processing metrics 
Rate Center Pooling Status 
p-ANI 

Customer Focus 

Change Orders 


See Appendix A for 2014 PA / NOWG Standing Agenda 
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Section 3.0 Customer Focus / Issues Log 
Customer Focus 


In 2014, at the monthly NOWG/PA meetings, the PA provided a report on 
customer focus items that they executed to help service providers and 
regulators. Customer focus items cover both contractual and non-contractual 
initiatives related to customer service. 


There were 116 customer focus items for Pooling and p-ANI in 2014. Customer 
focus items included, but were not limited to, the following: 


e Provided assistance to service providers on block donations 

e Provided time-saving and special reports for both service providers and 
regulators 

e Provided education and assistance on p-ANI resources 

e Provided service provider and regulator training 


PA/NOWG Issues Tracking Log 


There were no issues in 2014, as a result this log was discontinued. 
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Section 4.0 2014 PA Performance Survey Results 
4.1 PA Survey Ratings - Quantitative Analysis* 


The PA 2014 Performance Survey was completed by a total of 89 respondents. 
The respondents were comprised of 55 Industry and Other respondents and 34 
State Regulatory Commission respondents. 


Respondents provided a rating for one question in each section. Following are 
the aggregated response ratings: 


e Pooling Administrator (PA) 
e 81as Met 
e Oas Not Met 


e Pooling Administration System (PAS) 
e /5as Met 
e Oas Not Met 


e PA Website 
e 86as Met 
e 1as Not Met 


e Miscellaneous PA Functions 
e 82 as Met 
e Oas Not Met 


e PA Industry Activities 
e 58as Met 
e Oas Not Met 


e Overall Assessment of the PA 


e 87 as Met 
e Oas Not Met 


* The aggregated results do not include “N/A” responses. 


See Appendix B for 2014 PA Survey Metrics and Bar Charts, and Appendix C for 
2014 PA Survey Cover Letter and Performance Survey 
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4.2 PA Survey Written Comments 


The survey allowed respondents the opportunity to provide detailed written 
comments regarding their satisfaction or dissatisfaction with the PA’s 
performance in 2014. The majority of comments were positive, with only a few 
containing suggestions for areas of improvement. The NOWG reviewed all 
comments to determine if there was a common theme substantiated by multiple 
respondents. 


The following is a summary of written comments that were provided by survey 
respondents: 


e Outstanding praise for the PA staff was a consistent theme throughout the 
survey: 
o Responsive, Helpful, Prompt, Courteous 
Provided exceptional support 
A pleasure to work with 
Well-informed, Competent, Professional 
Excellent customer service 


oo0oo0o 


e Comments suggesting improvements were mostly isolated. Comments 
pertained to: 
o Communication of upcoming PAS enhancements 
o Timing of application processing and frequency of report generation 


The NOWG concluded that the written comments were not indicative of any 
consistent performance issues, and in many cases provided significant praise for 
individuals in the PA staff. Samples of the written comments received are 
provided below: 


“I work with Administrators in every state and they are all very helpful and 
a pleasure to work with.” 


“PAS representatives have always been helpful when | have questions. | 
am very pleased with all the PAS personnel | deal with.” 


“PSC had no problems with the PA and very much appreciate their help.” 


“We are very appreciative of the efforts of the PA staff. They are 
knowledgeable and have provided us some much needed support in 2014 
with special projects related to migration projects that required block 
modifications. This assistance upfront in the process enabled us to submit 
requests and avoid rework. Thank you to the entire PA staff for the 
outstanding support in 2014.” 


“Perfect customer service to the Carriers.” 
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“| feel the PA does a very good job in meeting the needs of all their clients. 
Every PA that | have come in contact with has always been responsive 
and professional. Working with PA’s | have always found them to be 
polite, businesslike & prompt. Just really dislike the once a week report 
delay for block returns. This really needs to change. Whenever we have 
dealt with the PA, they have been very helpful and courteous.” 


“| have always received prompt and courteous service from my PA. They 
always go over and above the level of assistance asked.” 


“When I need assistance from the PA I always get rapid, courteous, and 
helpful responses.” 


“The PA provided help/information requested and guidance with issues 
raised by this agency.” 


“Training | received was quite useful.” 


“| have had the most positive experience with the PA. All of my questions 
were answered in a timely fashion and in a positive manner.” 


See Appendix D for 2014 PA Survey Respondents and Appendix E for 2014 PA 
Survey Respondents’ Comments 
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Section 5.0 Operational Review 


The NOWG members met with the PA representatives in Concord, California on 
March 31 and April 1, 2015, to conduct the annual on-site operational review. 
During this review, the PA staff presented an overview and highlights of 2014 
activities. The presentation included the Pooling Administration operations, 
Pooling Administration System (PAS), external relations and training, change 
orders, pooling quality assurance, and regulatory compliance. 


Key highlights presented to the NOWG included: 
e Total Applications Processed in 2014: 
o Issued 139,181 Part 3s, which was the highest quantity of Part 3s 
processed since the start of Pooling 
o Assigned 50,440 thousands-blocks 
o Opened 3,381 NXX codes 
o Processed all but one application within 7 calendar days or less 
e Customer Support Desk received 1,118 calls and 100% were answered 
within one business day 
e Issued quarterly pooling tips 
e Authorized to reclaimed 21 blocks in 2014 
e Voluntary Donations 
o Excluded to Optional Rate Centers 
= PA sent out requests for donations in 86 rate centers and 
received donations in 66 rate centers 
o Optional and Single Service Provider Rate Centers 
= Voluntary donations requested in 7 rate centers and 
received donations in 4 rate centers 
e Trouble Tickets: 
o Opened and closed 6 new trouble tickets 
e Abandoned Blocks/Codes: 
o New block holders were located for 1,832 bocks 
o New code holders were located for 78 codes 
e Mass Modifications 
o 31,960 blocks modified from 104 spreadsheets entered into PAS 
o 2,740 codes updated in PAS from 23 spreadsheets from NANPA 
e Reporting: 
o The following reports were all submitted on time: 
=" 595 reports for the FCC, states, NANC, NANPA, and service 
providers 
= 62 ad hoc reports, all of which were provided in less than 
one business day (contract allows three business days) 
= 119 Contract Data Requirements List (CDRL) reports 
= 46 additional contract required reports 
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See Appendix F for 2014 PA Operational Review Presentation and Appendix G 
for 2014 PA Highlights 
Section 6.0 Pooling Administration System (PAS) 


The Pooling Administration System (PAS) was available 99.98% of the time in 
2014. The PA used a total of 4 hours and 39 minutes of scheduled, approved 
downtime to complete six maintenance events. PAS was also unavailable for a 
total of 2 hours and 3 minutes due to 3 instances of unscheduled downtime. 


Overall, the industry seemed satisfied with the performance of PAS in 2014; no 
criticisms or enhancement suggestions were provided in the industry responses 
to the performance survey. 


Building upon an effort started in 2013, the PA team continued development of 
the enhanced PAS in 2014. Multiple PA team members were responsible for 
writing requirements for all of the existing functions of PAS as well as new 
proposed functions and enhancements. The PA team produced more than 100 
documents totaling more than 1,300 pages in this effort. Each document was 
reviewed by a second PA team member, and in some cases, even a third PA 
team member. 


Throughout 2014, the PA team continued to update and modify requirements as 
the new PAS functionality was being developed, and tested the new functionality 
as it was completed. The PA team members created user testing procedures, 
tested extensively, and provided necessary feedback to the developers as 
needed. All of this was conducted while the PA team members continued to 
perform their daily tasks. Testing assignments were distributed throughout the 
team to accommodate daily workloads, vacation schedules, and illnesses of 
team members. The new enhanced PAS development and testing was 
completed in 2014, for release in January 2015. 
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Section 7.0 Change Orders 


In 2014 no new change orders were filed by the PA with the FCC. The PA 
change order process complies with FCC contractual requirements. The 
following change order, which was approved by the FCC in 2012, remained to be 
implemented in early 2015: 


e Enhancement of the FTP Interface with the Pooling Administration System 
(Change Order 24) was partially implemented on July 19, 2013. The 


remainder of this change order is scheduled to be implemented in January 
2015 with the roll out of the new PAS. 


See Appendix H for 2014 PA Change Order Matrix Log 
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Section 8.0 National Pooling Website 


The website maintained by the PA provides number pooling information to 
service providers and regulatory agencies. In 2014, the PA continued to keep 
the information current on the website. 


There were no new training videos created or published in 2014; however, the 

current videos on the PA website continued to be viewed regularly. The most 

popular video is “New to Pooling Quick Start,” which accounted for 50% of the 

views. There were 224 views of these videos in 2014. The training videos can 
also be downloaded from the PA website. 


Overall, the industry appeared satisfied with the website. 
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Section 9.0 p-ANI (pseudo-Automatic Number Identification) / RNA 
(Routing Number Administrator) 


The PA, in its second full year administering p-ANIs as the permanent Routing 
Number Administrator (RNA), continued a major work effort to reconcile p-ANI 
data discrepancies found during the initial reporting period. Discrepancies had 
been found where no p-ANI user reported on a p-ANI range that the assignor 
reported as assigned, and where more than one carrier reported on the same p- 
ANI range or part of a p-ANI range. Of the 4,561 p-ANI ranges initially identified 
with data discrepancies, only 58 p-ANI ranges remained to be resolved at the 
end of 2014. The RNA also worked diligently to resolve duplicate and 
overlapping p-ANI ranges that were identified. At the end of 2014, of the 287 
sets of overlapping p-ANI ranges found, only one set remained to be resolved. 


In addition to the data reconciliation work, the PA’s RNA functions in 2014 
included, but were not limited to, the following: 


e Processed 22,781 applications (Part 3s issued), with 99.99% of those 

applications processed on time 

Assigned 3,810 new p-ANI ranges 

Modified 9,780 existing p-ANI ranges 

Processed 9,124 returned p-ANI ranges 

Maintained the Routing Number Administration System (RNAS) with 

99.98% availability, as well as the p-ANI website 

e Continued to participate and hold a leadership position in the Emergency 
Services Interconnection Forum (ESIF), and participated in Industry 
Numbering Committee (INC) meetings to offer subject matter expertise 

e Processed carriers’ annual p-ANI reports and semi-annual forecasts 

e Completed and posted the 2014 p-ANI Activity and Projected Exhaust 
Report 

e Issued p-ANI Tips of the Month and Quarterly Tips to registered RNAS 
Users 


All of the work activities noted above, along with other RNA and customer service 
functions, were performed primarily by the two PA staff personnel who are 
dedicated resources for p-ANI Administration. 


Overall, the industry was satisfied with the PA’s RNA functions. This is apparent 
in the majority of responses received in both the quantitative sections and written 
comments sections of the NOWG’s RNA Survey. Following is a sample of 
comments received on the survey: 


“Always very responsive, helpful and fast.” 
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“When requesting new ranges of numbers RNA requires an FCC license 
for the market. The FCC website makes it difficult to find these licenses 
and [Names] are extremely helpful in getting the correct FCC licenses 
from the website.” 


“any questions or issues have been taken care of, explained. only during 
confirming our data but assistance was great.” 


“Thank you for providing the information and resources to the Local 
Government 9-1-1 PSAP / Coordinators. This information is most useful to 
our profession...” 


“[Names] are extremely helpful and very nice to work with! They go out of 
there way to assist me. | was new to requesting information from them in 
2014, and they were so very helpful each and every time | had questions. | 
have had only positive experiences with both of them!” 


“In our experience, the RNA representatives were courteous, professional 
and effective. Their responses were always timely with clear 
explanations...” 


The survey also contained comments from one respondent who suggested 
changes to the RNA’s processes, including the following: 


“The PA format still needs to include pANI assignments for PSAPs with 
multiple ESNs as well as PSAPs with different ESNs to different S/Rs...” 


“...PSAPs with wireless ESNs have different S/Rs based upon the 
transition from legacy to NG and we need to have the ability to accurately 
assign and document these multiple variations and ensure that over pANI 
provisioning does not occur thus ineffectively using the pANI pools.” 


See Appendix | for 2014 RNA Survey Respondents and Appendix J for 2014 
RNA Survey Respondents’ Comments 


16 


2014 PA Performance Evaluation Report 
May 27, 2015 


Section 9.1 RNA Survey Ratings - Quantitative Analysis* 


The following quantitative results were included in the PA Performance Review 
Methodology for 2014. 


The RNA 2014 Performance Survey was completed by a total of 17 respondents. 
The respondents were comprised of 16 Industry and Other respondents and 1 
State Regulatory Commission respondent. 


Respondents provided a rating for one question in each section. Following are 
the aggregated response ratings: 


e Routing Number Administrator (RNA) 
e 10as Met 
e 1as Not Met 


e Routing Number Administration System (RNAS) 
e 9as Met 
e 1as Not Met 


e RNA Website 
e 12 as Met 
e 1as Not Met 


e RNA Industry Activities 
e 9as Met 
e Oas Not Met 
e Overall Assessment of the RNA 


e 14 as Met 
e 1as Not Met 


* The aggregated results do not include the “N/A” responses. 


See Appendix K for 2014 RNA Survey Metrics and Bar Charts and Appendix L 
for 2014 RNA Survey Cover Letter and Performance Survey 
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Section 10.0 Conclusion and Recommendation 


The NOWG based its 2014 PA Performance Evaluation Rating on PA and RNA 
documentation, information collected, and observations throughout the review 
period year. For the 2014 performance evaluation rating, the NOWG considered 
PA and RNA activities that included interaction with the NOWG and NANC, 
active participation at INC and other industry forums, the PA’s ongoing 
consistency in addressing and resolving issues brought to their attention, and 
suggestions made by the NOWG throughout the calendar year. 


The 2014 survey results revealed a high level of client satisfaction with the 
continued professionalism and expertise exhibited by the PA personnel when 
performing their PA and RNA duties. The PA continued to demonstrate their 
ability to handle the large volume of block applications, while simultaneously 
completing special projects. 


The NOWG reviewed all aspects of the PA activities, as well as the feedback 
from service providers and regulators, and determined that the PA consistently 
performed very well in all areas of their responsibilities in 2014. As a result of the 
analysis, the NOWG gave the PA a “Met” rating for the 2014 performance year. 


The NOWG makes the following recommendations for the PA’s consideration in 
2015: 


e Continue to have internal training sessions with the PA and RNA personnel 
to ensure consistency in understanding the processes when responding to 
service providers and regulators. 


e Consider enhancing the current process for PAS Suggestions by creating a 
standardized form for suggestion submissions and distributing a notice to 
PAS users of the new PAS Suggestion form. 

e Propose an issue to INC to increase the frequency of requesting the 
Contamination Report from the NPAC for the purpose of approving or 
denying block returns. 


e Continue to provide high quality service to the industry. 
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